Comparing small-area mortality estimation models applied in subnational context

class: inverse, center, middle

# The performance of small-area mortality estimation models

## A simulation study

.large[Benjamin Schlüter]

.large[Bruno Masquelier]

<br/>

.large[Midi de la recherche| 18 Oct 2022]

---

# Context

Age-specific mortality rates (= mortality age schedule): input for demographic indicators

* Standardized mortality rates

* Life expectancy at birth, `$e^0$`

* Lifespan variation

Why do we need accurate mortality estimates by age and subnational areas?

* Document health inequalities

* Guide resource allocation

* Evaluate local policy measures

* Target areas most in need

<br/>

???

* __Within__ a country...

.center[<svg aria-hidden="true" role="img" viewBox="0 0 192 512" style="height:1em;width:0.38em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M176 432c0 44.112-35.888 80-80 80s-80-35.888-80-80 35.888-80 80-80 80 35.888 80 80zM25.26 25.199l13.6 272C39.499 309.972 50.041 320 62.83 320h66.34c12.789 0 23.331-10.028 23.97-22.801l13.6-272C167.425 11.49 156.496 0 142.77 0H49.23C35.504 0 24.575 11.49 25.26 25.199z"/></svg>  .highlight[Small population = unreliable mortality measurements]  <svg aria-hidden="true" role="img" viewBox="0 0 192 512" style="height:1em;width:0.38em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M176 432c0 44.112-35.888 80-80 80s-80-35.888-80-80 35.888-80 80-80 80 35.888 80 80zM25.26 25.199l13.6 272C39.499 309.972 50.041 320 62.83 320h66.34c12.789 0 23.331-10.028 23.97-22.801l13.6-272C167.425 11.49 156.496 0 142.77 0H49.23C35.504 0 24.575 11.49 25.26 25.199z"/></svg>]

---

# Stochasticity in death counts

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/fig_stoch_fr-1.png" alt="France 2007, female" width="80%" />
<p class="caption">France 2007, female</p>
</div>
]

.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/fig_stoch_be-1.png" alt="Belgium 2007, female" width="80%" />
<p class="caption">Belgium 2007, female</p>
</div>
]

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/fig_stoch_dist-1.png" alt="Namur district 2007, female" width="80%" />
<p class="caption">Namur district 2007, female</p>
</div>
]

.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/fig_stoch_muni-1.png" alt="Namur municipality 2007, female" width="80%" />
<p class="caption">Namur municipality 2007, female</p>
</div>
]

.center[
.highlight[Variability] & .highlight[Zeros]
]

???

* "Several models have been proposed to overcome these difficulties"

---

# Small-area mortality estimation models

.highlight[Models without area-level covariates]: endogeneity bias

.highlight[Models impose demographic regularity]: plausible shape

.highlight[Bayesian Hierarchical Models (BHM)]: leverage similarities in the data

<table>
 <thead>
  <tr>
   <th style="text-align:left;background-color: #7494A4 !important;"> Model </th>
   <th style="text-align:left;background-color: #7494A4 !important;"> Hierarchical </th>
   <th style="text-align:left;background-color: #7494A4 !important;"> Demographic regularity </th>
   <th style="text-align:left;background-color: #7494A4 !important;"> Name </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;background-color: #8FBC8F !important;"> Schmertmann (2021) </td>
   <td style="text-align:left;background-color: #8FBC8F !important;"> No </td>
   <td style="text-align:left;background-color: #8FBC8F !important;"> Penalizes deviations from HMD schedules </td>
   <td style="text-align:left;background-color: #8FBC8F !important;"> D-splines </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: #8FBC8F !important;"> Schmertmann &amp; Gonzaga (2016, 2018, 2020) </td>
   <td style="text-align:left;background-color: #8FBC8F !important;"> No/Yes </td>
   <td style="text-align:left;background-color: #8FBC8F !important;"> Standard mortality age schedule </td>
   <td style="text-align:left;background-color: #8FBC8F !important;"> TOPALS </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Congdon (2009) </td>
   <td style="text-align:left;"> Yes </td>
   <td style="text-align:left;"> None </td>
   <td style="text-align:left;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;background-color: #7F152B !important;"> Alexander et al. (2017) </td>
   <td style="text-align:left;background-color: #7F152B !important;"> Yes </td>
   <td style="text-align:left;background-color: #7F152B !important;"> Singular value decomposition </td>
   <td style="text-align:left;background-color: #7F152B !important;"> BHM </td>
  </tr>
</tbody>
</table>

???

* Schmertmann hierarchical and Congdon not considered in this presentation

* Bayesian models offer an additional tool to stabilize small-scale mortality estimation

* Bayesian modeling: natural framework for __hierachical__ modeling

* Admin1 = province and Admin2 = district/municipality: we care about Admin 2 mortality.

---
class: inverse, center, middle

# Resarch questions:

.left[
## 1. Compare the performance of the BHM with smoothing techniques encountered in small-area mortality estimation (TOPALS, D-splines) applied in a subnational context

## 2. Assess the capability of the BHM to reliably estimate quantities of interest in a subnational context
]

---

# The need for a simulation

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/fits_simu-1.png" alt="Namur municipality 2007, female" width="90%" />
<p class="caption">Namur municipality 2007, female</p>
</div>

---

# The need for a simulation

---

# The need for a simulation

---

# Methodology

.highlight[Simulation of a fictious country]

### Comparison metrics

.leftcol70[
* RMSE = `$\sqrt{\frac{1}{G}\sum^G_{x=1}(\hat{m}_x - m^{sim}_x)^2}$`
* Coverage = `$\frac{1}{G}\sum^G_{x=1}1[m_x^{sim} \geq l_x]1[m_x^{sim} < r_x]$`
]

.rightcol30[
`$m_x^{sim}$` known (simulated)
]

<br/>

computed across scenarios (see later)

???

* accuracy

* calibration

---

# Methodology

### Simulation's requirements

* Coherent mortality age schedules

* Realistic range of life expectancy at birth within the country

* Time dimension

* Mortality decreases over time

* Temporal stability of the best/worst performing areas

* At least two administrative levels

???

* Previous work in France and Germany showed that `$\Delta e^0 \leq 5-6$`

---
class: inverse, center, middle

# Simulation setup

---

# Simulation steps

1.Use Belgian spatial structure (10 provinces `$>$` 43 districts `$>$` 581 municipalities)

2.Associate Human Mortality Database (HMD) country `$i$` mortality to province `$p$`

3.Set `$l_x^{p, s} \equiv l_x^{i, s}$` for province `$p$`, sex `$s$` and HMD country `$i$`

4.Use Brass relational model `$logit(l_x^{a, p[a],s, t}) = \alpha^{a, p[a],s,t} + \beta^{a,p[a],s,t} \cdot logit(l_x^{p[a], s})$` for area `$a$` within province `$p$`, sex `$s$` and year `$t$` where areas are districts or municipalities

5.Assume and estimate

`$$\begin{bmatrix}\alpha^{p,male,t} \\ \beta^{p,male,t} \\ \alpha^{p,female,t} \\ \beta^{p,female,t} \end{bmatrix} \sim Multivariate~RW(1)~with~drift$$` 
for each province `$p$` corresponding to HMD country `$i$` over 15 years

---

# Simulation steps

6.Simulate 
 
`$$\begin{bmatrix}\alpha^{a,p[a],male,t} \\ \beta^{a,p[a],male,t} \\ \alpha^{a,p[a],female,t} \\ \beta^{a,p[a],female,t} \end{bmatrix} \sim N(                    \begin{bmatrix}\alpha^{a,p[a],male,t-1} + \hat{drift}^{p[a], male} \\ \beta^{a,p[a],male,t-1} + \hat{drift}^{p[a], male} \\ \alpha^{a,p[a],female,t-1} + \hat{drift}^{p[a], female} \\ \beta^{a,p[a],female,t-1} + \hat{drift}^{p[a], female} \end{bmatrix}, \hat{\boldsymbol\Sigma}_{rescaled}^{p[a]})$$` 
for each area over 10 years

7.Convert simulated `$logit(l_x^{a, p[a],s, t})$` into `${}_nm_x^{a, p[a],s, t}$`

8.Generate deaths with `$D_x^{a, p[a],s, t} \sim Poisson({}_nm_x^{a, p[a],s, t} \cdot E_x^{a, p[a],s, t})$`

---

# Provincial mortality

.leftcol65[
<img src="data:image/png;base64,#map_be.jpg" width="80%" style="display: block; margin: auto;" />
]

.rightcol35[
Mortality of 10 provinces

.center[=]

Mortality of 10 HMD countries in 2003, by gender
]

.leftcol65[
<img src="data:image/png;base64,#slides_files/figure-html/dev_brass-1.png" width="80%" style="display: block; margin: auto;" />
]

.rightcol35[

<br/>

.highlight[Brass relational model]

`$$logit(l_x^{area}) = a + b \cdot logit(l^{prov.}_x)$$`]

???

* Spatial structure of Belgium for our simulation <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M190.5 66.9l22.2-22.2c9.4-9.4 24.6-9.4 33.9 0L441 239c9.4 9.4 9.4 24.6 0 33.9L246.6 467.3c-9.4 9.4-24.6 9.4-33.9 0l-22.2-22.2c-9.5-9.5-9.3-25 .4-34.3L311.4 296H24c-13.3 0-24-10.7-24-24v-32c0-13.3 10.7-24 24-24h287.4L190.9 101.2c-9.8-9.3-10-24.8-.4-34.3z"/></svg> Admin1= 10 provinces, admin2= 43 districts or 581 municipalities

* 1st step: associate to each province a mortality age schedule from a country in the HMD in 2003 for both male and female

* a: level of mortality

* b: relationship between young and old mortality

---

# Correlation between Brass parameters

.leftcol65[
<img src="data:image/png;base64,#slides_files/figure-html/corr_brass_pars-1.png" width="80%" style="display: block; margin: auto;" />
]

.rightcol35[

<br/>

.highlight[Estimate multivariate random walks with drift on HMD country]
]

.leftcol65[
.highlight[For each area simulate]
`$$\begin{bmatrix}
a_t^f \\
b_t^f \\
a_t^m \\
b_t^m
\end{bmatrix} 
\sim N(
\begin{bmatrix}
a_{t-1}^f + \hat{drift}^f \\
b_{t-1}^f + \hat{drift}^f \\
a_{t-1}^m + \hat{drift}^m \\
b_{t-1}^m + \hat{drift}^m
\end{bmatrix} 
, 
 \hat{\boldsymbol\Sigma}^{rescaled})$$`
]

.rightcol35[

<br/>

* Drift <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M190.5 66.9l22.2-22.2c9.4-9.4 24.6-9.4 33.9 0L441 239c9.4 9.4 9.4 24.6 0 33.9L246.6 467.3c-9.4 9.4-24.6 9.4-33.9 0l-22.2-22.2c-9.5-9.5-9.3-25 .4-34.3L311.4 296H24c-13.3 0-24-10.7-24-24v-32c0-13.3 10.7-24 24-24h287.4L190.9 101.2c-9.8-9.3-10-24.8-.4-34.3z"/></svg> Temporal improvement (differs by province and gender)
 
* Scaling covariance matrices <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M190.5 66.9l22.2-22.2c9.4-9.4 24.6-9.4 33.9 0L441 239c9.4 9.4 9.4 24.6 0 33.9L246.6 467.3c-9.4 9.4-24.6 9.4-33.9 0l-22.2-22.2c-9.5-9.5-9.3-25 .4-34.3L311.4 296H24c-13.3 0-24-10.7-24-24v-32c0-13.3 10.7-24 24-24h287.4L190.9 101.2c-9.8-9.3-10-24.8-.4-34.3z"/></svg> Stability over time
]

???

* ... a and b over __10 years__ for both gender and then, used the Brass relational model to obtain its survival curves and hence, mortality age schedules. Repeat that process for each departments

---

# Scenarios

* Two administrative levels: .highlight[districts] & .highlight[municipalities]

<br/>

---

# Scenarios

* Two spatial structures: .highlight[hierarchy] & .highlight[random]

* Simulation leads to similarity for areas within the same province

* Random scenario: simulated mortality age schedules are randomly reshuffled across areas

---

# Scenarios

* Two levels of disparity: .highlight[realistic] & .highlight[high]

* Realistic scenario: simulated difference in `$e^0$` within the country is around 5 years
    
    * High disparity scenario: change the set of HMD countries associated to provinces and scale `$\hat{\boldsymbol\Sigma}_{p[a]}$` to double observed difference in `$e^0$`
    
<img src="data:image/png;base64,#map_be_alt.jpg" width="70%" style="display: block; margin: auto;" />

---
class: inverse, center, middle

# Simulation outputs

---

# Simulated life expectancy at birth

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/e0_real_nuts-1.png" alt="Districts, realistic disparity" width="95%" />
<p class="caption">Districts, realistic disparity</p>
</div>
]

.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/e0_real_lau-1.png" alt="Municipalities, realistic disparity" width="95%" />
<p class="caption">Municipalities, realistic disparity</p>
</div>
]

.pull-left[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/e0_ineq_nuts-1.png" alt="Districts, high disparity" width="95%" />
<p class="caption">Districts, high disparity</p>
</div>
]

.pull-right[
<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#slides_files/figure-html/e0_ineq_lau-1.png" alt="Municipalities, high disparity" width="95%" />
<p class="caption">Municipalities, high disparity</p>
</div>
]

???

* Realistic mortality decreases over time

* Realistic disparity in `$e^0$`

* Temporal stability in performance

---

class: inverse, center, middle

# Performance comparison

---

# Average RMSE

---

# Average 95% coverage

---
class: inverse, center, middle

# Indicators of interest in a subnational context

---

# Life expectancy at birth

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#e0_diag_dist.jpg" alt="Districts, realistic disparity" width="45%" />
<p class="caption">Districts, realistic disparity</p>
</div>

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#e0_diag_mun.jpg" alt="Municipalities, realistic disparity" width="45%" />
<p class="caption">Municipalities, realistic disparity</p>
</div>

* Correlation = 0.96 for both districts and municipalities
---

# Ranking according to life expextancy at birth

---

# Ranking according to life expextancy at birth

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#ranke0.jpg" alt="Districts, realistic disparity" width="90%" />
<p class="caption">Districts, realistic disparity</p>
</div>

* More than 80% of expected ranks in `$e^0$` are off by less than 5 position out of 43

* Ranking municipalities leads to much higher difference

* Wider credible intervals for `$e^0$`
    
    * Higher number of subnational areas

---

# Lifespan variation

<div class="figure" style="text-align: center">
<img src="data:image/png;base64,#sd.jpg" alt="Districts, realistic disparity" width="90%" />
<p class="caption">Districts, realistic disparity</p>
</div>

* Correlation = 0.84

---

# Conclusions

* Simulation offers an interesting setup to compare models over scenarios

* Simulation could be used to develop and test new small-area mortality estimation models

* In general, BHM has a better performance in terms of average RMSE and coverage than the two other models considered

* Performance of the BHM is negatively impacted by higher disparity in mortality age schedules present within the country

* Incorrectly specifying the hierarchy in the BHM does not significantly affect the performance

* BHM allows to reliably estimate demographic indicators at district level (life expectancy, ranking, lifespan variation) but metrics related to the overall distribution within the country are less reliable for municipalities (ranking)

---
class: inverse, center, middle

# Thank you for your attention !

.left[
<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M256 8C118.941 8 8 118.919 8 256c0 137.059 110.919 248 248 248 48.154 0 95.342-14.14 135.408-40.223 12.005-7.815 14.625-24.288 5.552-35.372l-10.177-12.433c-7.671-9.371-21.179-11.667-31.373-5.129C325.92 429.757 291.314 440 256 440c-101.458 0-184-82.542-184-184S154.542 72 256 72c100.139 0 184 57.619 184 160 0 38.786-21.093 79.742-58.17 83.693-17.349-.454-16.91-12.857-13.476-30.024l23.433-121.11C394.653 149.75 383.308 136 368.225 136h-44.981a13.518 13.518 0 0 0-13.432 11.993l-.01.092c-14.697-17.901-40.448-21.775-59.971-21.775-74.58 0-137.831 62.234-137.831 151.46 0 65.303 36.785 105.87 96 105.87 26.984 0 57.369-15.637 74.991-38.333 9.522 34.104 40.613 34.103 70.71 34.103C462.609 379.41 504 307.798 504 232 504 95.653 394.023 8 256 8zm-21.68 304.43c-22.249 0-36.07-15.623-36.07-40.771 0-44.993 30.779-72.729 58.63-72.729 22.292 0 35.601 15.241 35.601 40.77 0 45.061-33.875 72.73-58.161 72.73z"/></svg> .link-email[[benjamin-samuel.schluter@uclouvain.be](benjamin-samuel.schluter@uclouvain.be)]